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ABSTRACT 

The Substance Abuse and Mental Health Services 
Administration's Center for Substance Abuse Prevention (CSAP) is responsible 
for identifying and disseminating scientifically defensible knowledge about 
proven prevention models and principles to the substance abuse prevention 
field. This booklet is intended to assist prevention practitioners and others 
involved in the design, implementation, and evaluation of substance abuse 
prevention programs. This document provides the following: describes the 
evolution of the conceptual model that CSAP uses to provide a unifying 
framework for substance, abuse prevention; discusses the principles that 
provide the foundation for. understanding and defining research findings as 
scientifically defensible; and illustrates how CSAP applies. specific 
principles and criteria to research studies, program evaluations, and. 
scholarly efforts to identify scientifically defensible findings, prevention 
principles, and prevention models. The guide also provides specifics about 
the CSAP . methodology for selecting model programs and identifying 
scientifically defensible principles. It is one in a series of three products 
that together assist prevention practitioners, evaluators, state and local 
program administrators, policymakers., and funders in designing and assessing 
scientifically defensible programs. (Contains 25 references.) (GCP) 




Reproductions supplied by EDRS are the best that can be made 
from the original document. . 
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Although recent reports show a leveling or decrease in substance use among our nation's youth, drug abuse remains 
a problem in our country. There were 14.8 million current users of Illicit drugs In 1999. This figure represents 6.7 per- 
cent of the population 12 years and older. The 1999 National Household Survey also found Increases In illicit drug 
use among adults ages 18-25. Although the rates for those 26-34 years old and 35 years and older have not changed 
significantly since 1994, overall statistics Indicate that there Is still work to be done in preventing substance abuse. 

The Substance Abuse and Mental Services Health Administration's (SAMHSA) Center for Substance Abuse 
Prevention (CSAP) developed this series of products In response to the ongoing substance abuse problems. The three 
components in this series support CSAP's mission to provide resources that are based on science, with measurable 
outcomes, and designed to help community and state leaders formulate targeted programs. 

CSAP is committed to sponsoring, accumulating, and Integrating knowledge regarding scientifically defensible and 
effective prevention practices. The primary foci of each document in this series Is CSAP grantees, constituent organi- 
zations, and the communities these groups serve. 

We are pleased to release this guide to CSAP's conceptual framework and methodology for designing and assess- 
ing scientifically defensible programs for substance abuse prevention. The results reflect the findings of 10 years of 
CSAP-funded demonstration programs, as well as other advances in the design and evaluation of successful preven- 
tion strategies. These findings together lay the foundation for a new, empirically based approach to prevention pro- 
gramming. 

The booklet highlights the risk and protective factors that help determine an individual's vulnerability to substance 
abuse. It also examines CSAP's qualitative and quantitative strategies for evaluating existing substance abuse preven- 
tion programs and developing scientifically defensible best practices. 

This booklet is one in a series of products developed to help key stakeholders structure and assess scientifically 
defensible programs. It Is designed to serve practitioners and others involved in the development, implementation, 
and evaluation of substance abuse prevention programs as we work together on Innovative and effective solutions 
that respond to the unique needs of our individual communities. 



Joseph H. Autry III, M.D. 

(Acting) Administrator 

Substance Abuse and Mental Health Services 
Administration 



Ruth Sanchez-Way, Ph.D. 

Director 

Center for Substance Abuse Prevention 
Substance Abuse and Mental Health Services 
Administration 
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The Substance Abuse and Mental Health Services 
Administration's Center for Substance Abuse 
Prevention (CSAP) is responsible for identifying 
and disseminating scientifically defensible knowl- 
edge about proven prevention models and princi- 
ples to the substance abuse prevention field. 
Because there is a wide range of opinion about 
appropriate methodological criteria, standards, 
and expectations — ranging in rigor from requiring 
experimental studies to observing systematically 
to accepting clinical judgments — determining what 
constitutes that knowledge can be challenging. 

Over the past 1 0 years, CSAP demonstration 
programs have mirrored the field's growth in 
understanding what works in substance use pre- 
vention. Initially, programs implemented and test- 
ed prevention approaches based on practitioner 
observation. The realization, for example, that 
most people had little knowledge about the harm- 
ful effects of alcohol and illicit drugs led to the 
development of public information campaigns 
and school-based curricula designed to replace 
myths with facts. With increasing sophistication, 
this knowledge-centered focus enlarged to include 
innovative interventions based on theories derived 
from research in related fields (e.g., Bandura's 
social learning theory). More recently, data have 
become available from substance abuse preven- 
tion-specific program evaluations that provide 
empirical evidence about those strategies that suc- 
ceeded in changing knowledge, attitudes, and 
behavior. Those findings, many of which stem 
from CSAP-funded demonstration programs, have 
laid the foundation for a new empirically based 
approach to prevention programming — one that 
CSAP is challenging the field to use (Hansen and 
McNeal, 1996; Pandina, 1998). 



This booklet, Science-Based Substance Abuse 
Prevention: A Guide, is intended to assist preven- 
tion practitioners and others involved in the 
design, implementation, and evaluation of sub- 
stance abuse prevention programs. This document 
provides the following: 

■ Describes the evolution of the conceptual model 
that CSAP uses to provide a unifying framework 
for substance abuse prevention. 

■ Discusses the principles that provide the founda- 
tion for understanding and defining research 
findings as scientifically defensible. 

■ Illustrates how CSAP applies specific principles 
and criteria to research studies, program evalua- 
tions, and scholarly efforts to identify scientifical- 
ly defensible findings, prevention principles, and 
prevention models. 

The Guide also provides specifics about the 
CSAP methodology for selecting model programs 
and identifying scientifically defensible principles. 
It is one in a series of three products that together 
assist prevention practitioners, evaluators, state 
and local program administrators, policymakers, 
and funders in designing and assessing scientifical- 
ly defensible programs. The other two products 
are: 

■ Promising and Proven Substance Abuse 
Prevention Programs, a comprehensive compila- 
tion of both proven and promising interventions 
in an easy-to-scan grid organized by risk factor 
and domain; and 

■ Principles of Substance Abuse Prevention, a nar- 
rative and selected listing of proven and cited 
interventions organized by domain. 

State and federal agencies, local governments, 
and private foundations have become increasingly 
interested in funding substance use prevention 
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programs with measurable outcomes. The new 
emphasis on performance means that prospective 
funding applicants must demonstrate that the pro- 
grams they propose are both outcome-oriented 
and likely to achieve the outcomes predicted. 
Using scientifically defensible interventions can 
help prevention practitioners respond to demands 
for accountability and improve their capacity to 
provide effective services. 

CSAP's demonstration programs have been the 
testing ground for many of the most promising 
ideas for intervening with those at high risk for 
substance use. As a part of its knowledge develop- 
ment and application role, CSAP has accepted the 
responsibility for assessing those interventions and 
disseminating results to the field for replication 
and adaptation. In 1999, CSAP issued a primer on 
effective programs titled Understanding Substance 
Abuse Prevention, Toward the 21st Century: A 
Primer on Effective Programs that described eight 
model programs and briefly summarized the 
methodology guiding their selection. Given the 
increased emphasis on incorporating scientifically 
defensible principles and interventions in both 
new and existing programs, it is important that 
prevention practitioners understand the conceptu- 
al framework and criteria used in rating preven- 
tion interventions and attesting to their 
effectiveness. 

The CSAP Conceptual Framework: Risk 
and Protective Factors 

Theory and theoretical frameworks in the sub- 
stance use prevention field have been evolving 
over time, often through induction based on 
applied empirical research. Among the most 
important developments in substance abuse pre- 
vention theory and programming in recent years 
has been a focus on risk/protective factors as a 
unifying descriptive and predictive framework. 

Oisk Factors 

Put simply, one often tested and supported 
hypothesis derived from this framework is that the 
more risk factors a child or youth experiences, the 
more likely it is that she or he will experience sub- 
stance use and related problems in adolescence or 



young adulthood (Bry & Krinsley, 1990; Newcomb 
& Felix-Ortiz, 1992), Risk factors include biologi- 
cal, psychologicalA)ehavioral, and social/environ- 
mental characteristics such as a family history of 
substance use, depression or antisocial personality 
disorder, or residence in neighborhoods where 
substance use is tolerated. Researchers have also 
found that the more the risks in a child's life can 
be reduced — for example, by effectively treating 
mental health disorders, improving parents' family- 
management skills, and stepping up enforcement 
of laws related to the sales of illicit drugs to 
minors or to drinking and driving — the less vulner- 
able that child will be to subsequent health and 
social problems (Hawkins, Catalono, & Miller, 
1992), 

Protective Faotors aod Resilieoce 

Protective factors, such as solid family bonds and 
the capacity to succeed in school, help safeguard 
youth from substance use. Research has also 
demonstrated that exposure to even a substantial 
number of risk factors in a child's life does not 
necessarily mean that substance use or other prob- 
lem behaviors will inevitably follow. Many chil- 
dren and youth growing up in presumably 
high-risk families and environments emerge rela- 
tively problem-free. The reason for this, according 
to many researchers, is the presence of protective 
factors that reduce the likelihood that a substance 
use disorder will develop (Hawkins et al., 1992; 
Mrazek & Haggerty, 1994). The research on pro- 
tective factors explores the positive characteristics 
and circumstances in a person's life and seeks 
opportunities to strengthen and sustain them as a 
preventive device. Among these resilient children, 
protective factors appear to balance and buffer the 
negative impact of existing risk factors (Anthony & 
Cohler, 1987; Hawkins et al., 1992; Mrazek & 
Haggerty, 1994; Wolin & Wolin, 1995). From a 
substance abuse prevention perspective, protective 
factors function as mediating variables that can be 
targeted to prevent, postpone, or reduce the 
impact of use. 

Taken together, the concepts of risk and 
resilience enhance understanding of how and why 
youth initiate or refrain from substance use. 
Although not all risk and protective factors are 
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amenable to change — genetic susceptibility to sub- 
stance use, for example — research demonstrates 
that their influence can often be assuaged or 
enhanced. 

Domains 

Risk and protective factors exist at every level at 
which an individual interacts with others and the 
society around him or her. Clearly, the individual 
brings a set of qualities or characteristics to each 
interaction, and these factors act as a filter, color- 
ing the nature and tone of these interactions — 
whether positive or negative. One useful way to 
look at this interplay is to organize interactions by 
the six life or activity domains in which they 
chiefly occur. Based on more than 30 years of 
study, researchers have delineated specific subcat- 
egories of risk within each domain. They include 
the following: 



Domain Subcategory of Risk 



■ Individual 



■ Peer 

■ Family 

■ School/work 

■ Community 

■ Society/ 
environmental 



biological and psychological dispositions, 
attitudes, values, knowledge, skills, problem 
behaviors 

norms, activities, bonding 
function, management, bonding 
bonding, climate, policy, performance 

bonding, norms, resources, 
awareness/mobilization 

norms, policy/sanctions 



Research has also revealed that domains are not 
static in their impact, but interact with each other 
and change over time. As an individual develops, 
his or her perceptions and interactions with family, 
peers, schools, work, and community alter (Botvin 
et al., 1995; Donaldson, Graham, & Hansen, 

1994; Hawkins etal., 1992; Kumpfer, Molgaard, & 
Spoth, 1996). CSAP graphically depicts this more 
intricate set of relationships through its Web of 



Figure 1 

Web of Influence 



Teen 




School 




Substance 




Teenage 




Violent 


Pregnancy 




Dropout 




Use 




Suicide 


-> 


Crime 




Individual Risk and 
Protective Factors 




Society/ 
Environment* 
Related Risk and 
Protective Factors 



Family 

Risk and Protective 
Factors 



Community** Risk 
and Protective 
Factors 



School Risk and 
Protective Factors 



Peer Association 
Risk and Protective 
Factors 



‘Society/Environment: Refers to the total complex of external social, cultural, and economic conditions affecting a community or an individual. 
“Community: Refers to the specific geographic location where an individual resides and to the conditions within that particular area. 
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Universal programs (e.g., mass media, school- 
based health curricula): Target the general popula- 
tion. 

Selective programs (e.g., mentoring programs 
aimed at children with school performance or 
behavioral problems): Target those at higher-than- 
average risk for substance abuse. 

indicated programs (e.g., parenting programs for 
parents with substance abuse problems): Target those 
already using or engaged in other high-risk behaviors 
{such as delinquency) to prevent chronic use. 



Influence model (figure 1). The Web model illus- 
trates the complex series of interactions that 
occurs between the individual and the six external 
domains that can result in substance use and other 
problem behaviors. 

Institute of Medicine Preventinn Ciassificatinns 

Just as the Web of Influence can be used to illus- 
trate relationships among risk and protective fac- 
tors across domains, the Institute of Medicine's 
(lOM) prevention program classification system is 
useful in understanding the differing objectives of 
various interventions and matching them to the 
needs of a targeted population (Kumpfer et al., 
1997). 

The lOM system classifies prevention interven- 
tions according to the population they affect 
(Gordon, 1987). Universal interventions target 
general population groups without reference to 
those at particular risk. All members of a commu- 
nity benefit from a universal prevention effort, not 
just specific individuals or groups within a com- 
munity. Selective interventions target those who 
are at greater-than-average risk for substance use. 
Targeted individuals are identified on the basis of 
the nature and number of risk factors for substance 
use to which they may be exposed. Indicated 
interventions are aimed at individuals who may 
already display signs of substance use or abuse 
and are designed to prevent the onset of regular or 
heavy substance use. Together, the Web of 
Influence and the lOM classification system pro- 



vide a conceptual and organizational scheme for 
identifying risk groups and targeting outcomes. 

Issues in Defining Scientifically 
Defensible Knnwiedge 

Scientific inquiry stems from the need to under- 
stand the world at large. The strength of science 
and the scientific method is that it makes use of 
strictly defined, standardized procedures to deter- 
mine how events are causally related. As science 
improves its methods, levels of certainty about the 
nature and extent of cause-and-effect relationships 
increase and more is understood about the 
resources and effort required to achieve specific 
changes in existing relationships. Using the scien- 
tific method more systematically to identify knowl- 
edge also fosters recognition of the diversity of 
approaches involved in implementing prevention 
programs and extracting data. 

Uifferent Ways of Knowing 

A key CSAP objective is to ensure that the inter- 
ventions it recommends to the field are out- 
growths of rigorous evaluation and are described 
in credible terms, comprehensible to the multiple 
target audiences involved in substance abuse pre- 
vention. Emerging out of the cultural, geographic, 
and strategic diversity of high-risk population pro- 
gramming is a recognition of the different ways in 
which people know or understand that a program 
is having an impact on youth, families, and com- 
munities. 

Like good medicine, the practice of prevention 
is both an art and a science. In assessing preven- 
tion programs as a whole and attempting to under- 
stand whether the strategies and interventions 
used are exerting an impact and how they affect 
human behavior, it is critical to consider both 
quantitative and qualitative evidence. Quantitative 
data supply the raw material for the extensive sta- 
tistical analyses that lend scientific credence to 
program results. Qualitative data provide the rich, 
descriptive information needed to explain the 
effects of program interventions. 
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Data types and Researcti Strategies 

Although much discussion of knowledge focuses 
on the results of quantitative outcome evaluations, 
qualitative information can also be extremely use- 
ful even if it is not always amenable to strict out- 
come evaluation. Qualitative data may describe 
program process or identify contextual mediating 
variables that affect outcome results. Such process 
information embellishes findings from programs, 
providing an enhanced understanding of program 
results. When researchers and the field, in general, 
ignore qualitative data, valuable information can 
be lost. 

Reviews of qualitative information can produce 
credible findings and recommendations. For 
example, expert consensus panels convened by 
many government agencies (e.g.. Center for 
Substance Abuse Treatment, Food and Drug 
Administration, National Institute on Alcohol 
Abuse and Alcoholism) and private organizations 
review and use both qualitative and quantitative 
data to reach conclusions and formulate recom- 
mendations affecting the health and well-being of 
the nation as a whole. CSAP has developed 
Prevention Enhancement Protocol Systems (PEPS), 
which convene expert consensus panels to identi- 
fy what is known about various dimensions of 
substance use prevention. PEPS panels also exam- 



ine qualitative data from prevention practice cases 
in their efforts to assess interventions such as 
tobacco prevention/cessation and family-oriented 
prevention. 

Date Dalleotioa Teotiiilgiies 

Figure 2 identifies the numerous data collection 
techniques that are used to gain knowledge in the 
substance abuse field. The techniques are mapped 
onto a pyramid to provide some idea of the pro- 
portion of total information available through par- 
ticular techniques that have been sufficiently 
implemented to yield credible findings. 

Paradoxically, as indicated in the figure, the 
more traditionally accepted and more quantitative 
scientific approaches represent a small proportion 
of data collection efforts, yet the information 
derived from such studies constitutes a significant 
portion of the formal knowledge base. Although 
the tendency is to ascribe more validity to quanti- 
tative efforts, CSAP recognizes that important con- 
tributions to the prevention knowledge base are 
also made by qualitative studies. Independent of 
how quantitative or qualitative they are, when 
findings are supported by sound scientific princi- 
ples, they can be used legitimately and effectively 
to construct and implement prevention interven- 
tions. 



Figure 2 

Application of Specific Principles and Criteria to Research Findings 
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Application of Specific Principles and 
Criteria to Research Findings 

Research studies and the findings they produce 
vary in credibility. The level of observed credibili- 
ty of research findings regarding the causes of 
intervention program effects hinges on whether 
the methods employed provide at least a reason- 
able means of assessing change over time attribut- 
able only to the program. This simple criterion has 
a number of components that can be applied to 
assess the credibility of research. The development 
of the High Risk Populations (HRP) Databank, a 
previous CSAP endeavor that critically reviewed 
the HRP Demonstration Grants, provides an exam- 
ple. 

The HRP Findings Bank was an evaluation- 
oriented information system with a comprehen- 
sive, unifying framework. The Findings Bank 
consisted of three primary components: 

■ Descriptive information (e.g., location, number 
and type of sites, target group demographics) 
about CSAP's high-risk populations grants; 

■ Compilation of interventions and outcomes pro- 
duced by the grants; and 

■ Compilation of interventions and outcomes pro- 
duced by similar, non-CSAP projects. 

The Findings Bank allowed users to relate findings 
from demonstration programs to outcome mea- 
sures and to the data that support the findings. 

Nine criteria were used by groups of trained, 
expert evaluators to assess the rigor of grantee 
program evaluations. Six criteria were used to 
rate various aspects of methodological rigor (1 
through 4, 6, and 7), one was used to rate fidelity 
(5), one for utility (9), and one for overall integrity 
and credibility (8). 

Viewed in an alternative fashion, criteria 1 
through 3 rated the design or planning of the 
study. They encompassed theoretical bases, sam- 
ple design, and the outcome measures chosen. 
These three criteria pertained to the planning of 
both quantitative and qualitative research. 

Criteria 4 through 6 related to execution of the 
study; in other words, how it was implemented. 
Criterion 5 was a hybrid, addressing both fidelity 
of program/intervention implementation and the 
design issue of dosage of program/intervention. 



Criteria 7 and 8 took the form of summary eval- 
uations. Criterion 7 captured how well the total 
design and execution ruled out alternative 
hypotheses, while criterion 8 reflected overall 
confidence in results inspired by methodological 
aspects of study design and execution. Criterion 9 
addressed the utility of study results. Considered 
together, the nine criteria provide one scheme for 
evaluating all aspects of a quantitative or qualita- 
tive intervention. 



PrograiB Mew firfteria 

1 . Theory: the degree to which findings are 
grounded in sound theory, reflect clearly 
stated hypotheses, and are operationally 
relevant 

2. Sampling strategy and implementation: the 
quality of sampling design and implemen- 
tation and strength of evidence concerning 
sample quality (e.g., data on attrition) 

3. Measures: operational relevance, psycho- 
metric quality of measures used in the eval- 
uation, and quality of supporting evidence 

4. Data collection: quality of data collection 
implementation (e.g., amount of missing 
data) 

5. Fidelity of interventions: evidence of high- 
fidelity implementation of program, as 
designed, and sufficiency of dosage (e.g., 
duration, intensity, frequency) to effect pos- 
itive change 

6. Analysis: appropriateness and adequacy of 
statistical techniques used in analysis 

7. Plausible threats to validity: degree to 
which evaluation design and implementa- 
tion address and eliminate reasonable alter- 
native hypotheses about program effects 
and warrant strong causal attributions 

8. Integrity: overall level of confidence in 
project findings based on the research 
design and implementation 

9. Utility: strength of findings and strength of 
evaluation to determine if findings were 
consistent with respect to expectations or 
predictions from theory 
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5 strong confidence 
4 = confident 
3 = some confidence 
2 = little confidence 
1 = no confidence 




For the review of CSAP's HRP demonstrations, 
pairs of trained evaluators rated each of the nine 
criteria on a 5-point scale. Of particular impor- 
tance was the rating of integrity, on which review- 
ers were required to come to consensus. This 
rating reflected how much confidence reviewers 
had that the intervention alone was responsible for 
yielding the findings observed. Confidence was 
derived from the quality of the intervention imple- 
mentation as well as the design of the evaluation 
study and how well the evaluation was conduct- 
ed. For example, when problems in intervention 
implementation and research design and execu- 
tion were minimal, reviewers assigned integrity 
ratings reflecting strong confidence in the findings 
(i.e., 5). 



When a few problems were encountered — for 
example, attrition was modest, intervention imple- 
mentation was solid, and analysis was accept- 
able — an integrity rating of 4 (i.e., confident) was 
given. 

An integrity rating of 3 reflected some confi- 
dence in resultant data. This rating was often used 
when program characteristics were strong enough 
to inspire some confidence, but because of imper- 
fect implementation of the program, moderate 
attrition rates, data analyses that were not compre- 
hensive, uncorrected differences between treat- 
ment and comparison samples, or secular events 
that contaminated samples, confidence was not 
sufficient to assure reviewers that the results were 
wholly attributable to the program intervention. 

If reviewers had little confidence in an experi- 
mental study in which there was high and differ- 
ential attrition, the integrity rating received by 
such a study was 2 — little confidence. If attrition 
was egregiously pronounced, the study rating may 
have dropped to 1 — no confidence. 

Because well-designed studies can be imple- 
mented poorly, well-implemented programs can 
be evaluated poorly, and findings can be overstat- 
ed, it is important to use criteria capable of expos- 
ing these problems. Although overstated findings 
make no positive contribution to the knowledge 



Figure 3 

Sample Application of Scientific Criteria to an Intervention 
Summary Matrix for Grant ZZZ 
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base, from the perspective of the prevention pro- 
gram designer, policymaker, and funder, negative 
findings from well-implemented, rigorously evalu- 
ated interventions have enormous value because 
they identify interventions that do not appear to 
work in naturalistic settings. When objectively 
reported, they save programs from expending 
resources in the application of Ineffective 
approaches. 

The last rating of utility captured data patterns. 
Ideally, the field would want to adopt or adapt 
interventions that yield consistent changes across 
domains affected by the Intervention. For exam- 
ple, program X could demonstrate a dramatic 
change on a specific targeted behavior such as 
marijuana use. In response, prevention practition- 
ers in a community where marijuana use was 
increasing might rush to adopt program X without 
considering changes across other outcome 
domains such as skyrocketing underage use of 
alcohol (drug substitution) or plummeting self- 
efficacy. Reviewers used the utility rating to gauge 
the pattern of outcomes and thereby avoid focus- 
ing on isolated positive or negative outcomes. 

Hypothetical Review lllostratieg applicatioe of 
Criteria 

Figure 3 depicts a hypothetical review. In the fig- 
ure, program ZZZ has an overall rating of 3. The 
program was rated well on many of the criteria; 
however, reviewers did not think that program 
control and treatment groups were comparable 
prior to the intervention, hence the evaluation of 
2 for sampling strategy and implementation. 
Furthermore, the statistical analyses testing differ- 
ences between the groups after the Intervention 
did not attempt to control for what might have 
been meaningful pretest differences. For that rea- 
son, analyses received a 2. In addition, the age 
group differences observed on outcome measures 
were large and not predicted by theory. However, 
this intervention, along with a number of others, 
showed differences In youth's knowledge of the 
harmful effects of substance use as a result of par- 
ticipating in classroom-based drug education, an 
outcome that, given the measurement protocols, 
could not be attributed to any event or occurrence 
other than the intervention. Therefore, despite 



mixed confidence, this intervention was considered 
promising and could be cited as corroborative evi- 
dence of the effectiveness of grant ZZZ in altering 
youth's knowledge about the harmful effects of 
substance use. 

In addition to applying the criteria to the HRP 
demonstration grants, which include both the High 
Risk Youth (HRY) programs and programs for 
Pregnant and Postpartum Women and their Infants, 
CSAP is using them in a slightly modified form to 
extract findings from the Community Partnership 
Program. Likewise, CSAP's National Registry of 
Effective Prevention Programs (NREPP) — an ongo- 
ing repository of scientifically defensible guidance 
for the substance abuse field — has Incorporated the 
criteria in the 15 dimensions used In its consensus 
process when assessing the quality of a particular 
prevention program. These 15 criteria are an 
expansion of the original nine criteria described on 
page 6 of this guide. For a full review of these cri- 
teria, please refer to the NREPP Web site at 
www.preventlonreglstry.org. NREPP has evolved 
from the HRP Findings Bank and is available to 
review programs funded from any source, not only 
from CSAP. 

The CSAP Approach To Bdentifyinp 
Scientifically Defensible Prevention 
Interventions 

The CSAP HRP Databank review used a qualita- 
tive meta-analytic technique, one of two types of 
meta-analytic techniques currently favored to 
organize information and extract defensible princi- 
ples and practices. In the past, literature reviews 
were frequently analyzed to determine whether 
substance abuse prevention programs worked. 
Meta-analysis offers a major refinement on that 
approach. 

In a literature review, researchers scrutinize and 
critique original papers, determine the merit of 
specific items, and then integrate the findings In 
discussions with each other or alone. 

Occasionally this thinking results in the proposi- 
tion of a critical experiment that, if performed, 
would shed light on the true phenomenon under 
scrutiny. More often, the result is a simple summa- 
ry of the field, with the authors' conclusions sup- 
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ported by the amount and consistency of the data 
assembled. 

Literature reviews make important contributions 
to understanding the state of the art in the field 
and help identify gaps in the knowledge base that 
should be filled. However, because literature 
reviews are, for the most part, essentially narrative 
and subjective, use few samples, and lack scientif- 
ic rigor, they cannot respond adequately to ques- 
tions about the efficacy of substance abuse 
prevention programs (Tobler, 1994, p. 343). 

Meta-Analysis 

To address concerns about program effects, CSAP 
turned to meta-analysis, a conceptual approach 
that reviews the results of the full range of primary 
research, including both published and unpub- 
lished, experimental and quasi-experimental stud- 
ies of programs that succeeded and programs that 
failed. Meta-analysis uses qualitative as well as 
quantitative methods to produce aggregated 
results from multiple programs that can be used to 
study relationships and test hypotheses (Cook et 
al., 1992; Light & Pillemer, 1984; Tobler, 1994, 
p. 357). Because meta-analysis allows researchers 
to use results from small studies, from studies that 
are quasi-experimental in design, and from studies 
with incomplete information in some areas, it 
offers important advantages when evaluating sub- 
stance abuse prevention programs (Tobler, 1994, 
p. 350). In applying meta-analysis, CSAP makes a 
distinction between the two types. While both 
forms rely on qualitative judgments as well as 
quantifying information, they differ in the extent to 
which they rely on translating initially observed 
outcomes into hard estimates of effect sizes. 

Quantitative meta-analysis systematically codes 
the results from each study for every variable that 
may influence program outcomes and converts 
those results to a standardized score or effect size 
(Tobler, 1994, p. 345). The end product of the 
analysis consists of a single best quantified esti- 
mate of effect for a specific intervention or the 
impact of a single contextual factor. Standardi- 
zation enables meta-analysis to compare programs 
with different sample sizes, and if effect sizes are 
consistent and positive, to aggregate them and 
achieve statistical significance. Because quantita- 
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tive meta-analytic techniques focus on specific 
effects, they are extremely useful in identifying 
consistently moderate or large outcomes, and then 
in relating these observations to specific program 
characteristics. Quantitative techniques are also 
useful in deriving prevention principles and identi- 
fying specific prevention interventions that are most 
closely related to consistently favorable outcomes. 

Like any other procedure, meta-analysis is not 
without potential hazards (e.g., relying solely on 
journal publications, averaging equally over differ- 
entially important measures). Recent quantitative 
meta-analytic efforts, including Tobler's and 
Stratton's (1 997) analysis of the effectiveness of 
school-based substance use prevention programs 
and CSAP's National Center for the Advancement 
of Prevention meta-analyses of the Correlates of 
Marijuana Use and the Correlates of Alcohol and 
Tobacco Use, have advanced the knowledge base 
considerably. 

Qualitative meta-analysis is based on programs 
as a whole. As an example, in the CSAP HRP 
Databank review, trained, expert reviewers used 
standard instruments to evaluate individual pro- 
grams on the basis of source documents in the 
form of project final reports, as well as articles 
published in journals, and come to consensus 
about their credibility. Inclusion of source materi- 
als overcame biases that may have been inherent 
in using journal articles only. 

Qualitative techniques are not limited to main 
effects models of causation but are robust across 
interactions and nonlinear models since judg- 
ments of effectiveness are based on the integration 
of implementation, evaluation, and findings within 
the context of the program. While ratings may be 
attached to program efforts (e.g., integrity, utility), 
they are composites weighted by the judgment of 
expert evaluators and not the product of decon- 
structed program characteristics averaged across 
multiple programs or interventions. Systematizing 
procedures and categorizing outcomes across pro- 
grams via qualitative meta-analytic technique not 
only offers decided advantages to traditional litera- 
ture review efforts, but also ensures that both qual- 
itative and quantitative information is scrutinized 
so that researchers, policymakers, and program 
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designers have access to needed outcome and 
process data. 

Distinctions Between Quantitative and 
Quaiitative Techniques 

Like quantitative meta-analytic techniques, quali- 
tative procedures use rules for inclusion of infor- 
mation. In general, these rules mirror the nine 
criteria described previously in reference to the 
HRP Databank. No matter who conducts the 
review, research protocols are screened for quality 
of design and implementation. What may differ is 
the set of procedures, level of data disaggregation, 
population of studies eligible for inclusion, and 
rigor of the review itself. 

While both quantitative and qualitative tech- 
niques have important roles to play, their purposes 
are different. The more quantitative techniques are 
useful in identifying interventions and contextual 
factors that influence effectiveness, whereas quali- 
tative techniques are most useful in identifying 
effective programs and models of intervention. As a 
result, qualitative techniques can be used to identi- 
fy both successful program models and prevention 
principles. Both quantitative and qualitative tech- 
niques can be supported by field observation and 
careful review of complete program documenta- 
tion (e.g., process analyses). Also, both are valu- 
able in developing and deriving principles 
regarding successful program implementation. 

Resources for Ideotifymg ScieotificoHy 
Defeosible Preveotioo Priociples aod 
Programs 

CSAP uses both qualitative and quantitative meta- 
analyses, as well as findings from critical 
overviews of published research and expert con- 
sensus procedures, to identify scientifically defensi- 
ble prevention principles and programs. Currently, 
those principles and programs are described in two 
CSAP publications. Principles of Substance Abuse 
Prevention and Promising and Proven Substance 
Abuse Prevention Programs. Following are descrip- 
tions of the specific methods CSAP employed to 
identify the scientifically defensible principles and 
programs in both documents. 



Qualitative Meta-Analyses 

CSAP culled several well-implemented critical 
reviews to identify credible scientifically defensi- 
ble principles and programs, including the follow- 
ing CSAP-sponsored efforts: 

■ Environmental Strategies for Substance Abuse 
Prevention: Analysis of the Effectiveness of 
Policies to Reduce Alcohol, Tobacco, and Illicit 
Drug Problems; 

■ The Role of Education in Substance Abuse 
Prevention (Implementation Guide); 

D The Role of Information Dissemination and Mass 
Media in Youth-Oriented Prevention 
(Implementation Guide); 

■ The Role of Problem Identification and Referral 
in Youth-Oriented Prevention (Implementation 
Guide); and 

■ A Review of Alternative Activities and Alternative 
Programs in Youth-Oriented Prevention (CSAP 
Technical Report 13, 1996). 

Expurt Panels 

In addition to information derived from critical lit- 
erature reviews, CSAP also incorporated informa- 
tion derived from expert consensus procedures. As 
part of the consensus process, consensus panel 
members assessed implementation for fidelity and 
reviewed research for rigor. Outcomes and review- 
er confidence in those outcomes were rated using 
relatively objective, standard methods. Expert con- 
sensus resources included the following: 

■ National Structured Evaluation (NSE): The NSE 
was one of the first expert consensus reviews of 
prevention programs. Using standard procedures 
and measures, the NSE reviewed and rated the 
level of methodological rigor of substance use 
prevention programs implemented through 
1991 . The NSE effort included federally funded 
programs as well as programs sponsored by uni- 
versities, foundations, and state or local govern- 
ments. When the analysis was complete, 10 
percent of the studies reviewed met or exceeded 
the criteria set for moderately rigorous studies 
(i.e., a rating of 3). 

■ William B. Hansen & L. A. Rose (1997): In his 
work. Dr. Hansen continued to use techniques 
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employed by the NSE to identify and review sub- 
stance abuse prevention program evaluations 
and field studies that reflected effective program- 
ming. Data from individual projects were used to 
derive more general principles concerning pro- 
gram effectiveness. 

a National Institute on Drug Abuse (NIDA, 1997): 
NIDA produced a review document titled Drug 
Abuse Prevention for At-Risk Individuals that cit- 
ed NIDA-sponsored substance abuse prevention 
programs viewed as effective and offered general 
recommendations about key elements of effec- 
tive interventions. 

■ Lawrence W. Sherman et al. (1 997): Dr. Sherman 
and colleagues conducted a methodologically 
rigorous review and rating of U.S. Department of 
Justice programs modeled after the NSE 
approach. The authors also evaluated grant and 
funding mechanisms for adequacy. Principles 
pertaining to effective strategies were derived 
and presented as key findings. 

B PEPS: Over the past several years, CSAP has 
sponsored the PEPS, a knowledge development 
effort that convenes a panel of experts to review 
the literature in a specific area using a strict evi- 
dentiary procedure to evaluate and either 
include or exclude individual research findings. 
Findings are then assessed for consistency of 
valence and magnitude to arrive at a defensible 
conclusion. 

B National Evaluation of the Community 

Partnership Prevention Grant Program (2000): 
Extensive cross-site and other evaluations of the 
CSAP Community Partnership Program have 
yielded considerable information about commu- 
nitywide prevention efforts and have identified 
specific prevention strategies that produce mea- 
surable reductions in substance use. 

B The CSAP HRP Databank review: This review of 
the HRP demonstration grants program was 
designed to identify credible evidence of pro- 
gram effectiveness. An expert consensus process 
evaluated each final report filed through 
December 1995 on the criteria defining program 
credibility. In addition, they rated utility of study 
information for each program and weighed the 
consistency of results in determining the evalua- 




tion of program effectiveness. Like the NSE, 
approximately 10 percent of program evalua- 
tions met the criteria set for moderate rigor. Of 
those, an estimated 2 percent of studies reviewed 
met the more stringent standards that CSAP 
developed for identifying well-implemented, 
solidly evaluated, effective model programs. 

Quantitative Meta-Anaiyses 

The most stringent coding and rating protocols can 
be followed when there are a wealth of studies with 
concordant data. As the discipline of substance 
abuse prevention continues to evolve, quantitative 
efforts like these will become more commonplace. 
CSAP used data from the following quantitative 
meta-analyses to identify current scientifically 
defensible principles and programs, as follows: 

B Tobler's meta-analyses of school-based preven- 
tion programs (1986, 1992). 

B Tobler & Stratton, 1997. 

B CSAP meta-analysis: CSAP initiated a series of 
large-scale meta-analyses, each focusing on a 
specific topic area relevant to substance use and 
substance use prevention (e.g.. Correlates of 
Marijuana Use Among Youth, Correlates of 
Alcohol and Tobacco Use Among Youth and 
Meta-Analysis of the Effectiveness of School- 
Based Programs). 

CSAP accepts data from these qualitative and 
quantitative sources as credible, relying on the 
review processes implemented and the expertise 
of the review panels convened to extract impor- 
tant research findings. With these data as building 
blocks, it is possible to generalize across agencies 
and the time periods in which the data were col- 
lected to identify replicable model programs and 
scientifically defensible principles and interven- 
tions. 

CSAP has developed a series of comprehensive 
data matrices (Promising and Proven Substance 
Abuse Prevention Programs) that organizes inter- 
ventions by risk and protective factors, age, strate- 
gy, lOM classification, original program name, and 
source citation. This inclusive effort includes pro- 
grams supported by government agencies, includ- 
ing CSAP, National Institute on Drug Abuse 
(NIDA), National Institute on Alcohol Abuse and 
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Alcoholism, the U.S. Department of Education, 
and the U.S. Department of Justice, that have 
undergone strenuous reviews (such as the CSAP 
model programs) or have been cited by experts in 
the field as examples of particular intervention 
strategies. Interventions entered into the matrix 
vary in the levels of rigor assigned by reviewers. 

For example, the CSAP model programs incorpo- 
rated in the matrix represent the highest level of 
rigor and credibility. Interventions from other CSAP 
programs, such as the Community Partnerships, 
which may have more moderate levels of credibili- 
ty, are included because careful evaluation attests 
to their ability to produce positive outcomes. 
Interventions from other federal agency programs 
are included because their sponsors have identified 
them as noteworthy through their own evaluative 
processes. The matrix is a work in progress that 
will continue to expand as findings from other 
promising interventions become available. As the 
information in the data matrix increases, the inter- 
ventions cited will provide a broader range of 
interventions and more specific guidance for 
selecting those most likely to prove effective with 
particular populations and in particular domains. 

Also available is a second, simpler compilation 
of scientifically defensible principles called 
Principles of Substance Abuse Prevention. As its 
name implies, this brief booklet organizes princi- 
ples and interventions by domain and links them to 
the prevention strategies identified in the Substance 
Abuse Prevention and Treatment block grant. 
Service providers can refer to Principles for ideas 
about what works in substance use prevention, to 
identify proven interventions in a particular 
domain, and to justify the use of one or more inter- 
ventions in a program. Because each principle and 
intervention is cited, it is relatively easy to locate 
the full article for more detailed information. 
Evaluators, grant reviewers, and policymakers may 
also find it useful as a quick overview of the cur- 
rent state of the art in prevention programming and 
as a tool useful in ascertaining whether a program 
is employing scientifically defensible principles 
and interventions. 
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Antisocial and other problem behaviors: Can describe behavior-related problems (e.g., poor conduct and impulsive- 
ness), behavior-related disorders (e.g., attention deficit hyperactivity disorder), or both. 

Approach: A set of prevention strategies that typifies a program and can be used in an intervention setting without 
adopting the entire program. 

Assignment: The process by which researchers place study subjects in an intervention, control, or comparison group. 
Experimental design studies randomly assign study subjects to both intervention and control conditions. In quasi- 
experimental studies, study subjects are nonrandomly assigned to intervention and comparison conditions. Random 
assignment increases the likelihood that the intervention and control groups are equal or comparable and have simi- 
lar characteristics. 

Attrition: An unplanned reduction in the size of a study sample caused by participants dropping out of the evaluation 
(e.g., they moved away from the study location). 

Behavior-related disorder: A specific behavioral problem that occurs in persistent patterns and characteristic clusters 
and causes clinically significant impairment. 

Behavior-related problem: A behavioral problem that is isolated or intermittent, is not part of a persistent behavior 
pattern, and varies in severity and seriousness of its consequences. 

Community: A group of individuals who share cultural and social experiences within a common geographic or politi- 
cal jurisdiction. 

Community-based approach: A prevention approach that focuses on the problems or needs of an entire community, 
be It a large city, small town, school, worksite, or public place. 

Community readiness: The degree of support for or resistance to identifying substance use and abuse as significant 
social problems in a community. Stages of community readiness for prevention provide an appropriate framework for 
understanding prevention readiness at the community and state levels. 

Community tolerance: Community norms that view problematic behavior as socially acceptable or actively encour- 
age it. 

Conduct disorder: A behavior-related disorder that has a repetitive and persistent pattern of violating the basic rights 
of others or major age-appropriate societal norms or rules. The disorder can include aggression to people and ani- 
mals, destruction of property, deceitfulness or theft, and serious violation of rules. 

Construct: An attribute, usually unobservable (e.g. educational attainment or socioeconomic status), that is represent- 
ed by an observable measure. 

Control group: In experimental evaluation design, a group of participants that is essentially similar to the intervention 
group but is not exposed to the intervention. Participants are designated to be part of either a control or an interven- 
tion group through random assignment. 

Credibility of findings: Derives from the quality of intervention implementation plus the methodological rigor of the 
research. When both are high, findings are attributable to the intervention and therefore have high credibility. 

Data: Information collected according to a methodology using specific research methods and instruments. 

Data analysis: The process of examining systematically collected information. 

Design: An outline or plan of the procedures to be followed in scientific experimentation and research studies to 
reach valid conclusions. 
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Documentation: Entails keeping records, collecting data, and making observations to obtain specific kinds of infor- 
mation, such as the rates of alcohol-related problems, consumption, and sales. 

Effect: A result, impact, or outcome. In evaluation research, attributing an effect to a program or intervention 
requires establishing, through comparison, a logical relationship between conditions with and without the program 
or intervention. 

Effective: The preponderance of research or program findings is consistent, positive, and clearly related to the inter- 
vention. 

Environmental factors: Those factors that are external or perceived to be external to an individual but that may none- 
theless affect his or her behavior. At a narrow level these factors relate to an individual's family setting and relation- 
ships. At the broader level, these refer to social norms and expectations as well as policies and their implementation. 

Evaluation: The analysis of data obtained through documentation to assess the operation or impact of a policy, pro- 
gram, intervention, or procedure. 

Evaluation research: A set of procedures to determine the effectiveness of an intervention program. 

Experimental design: A research design involving random selection of study subjects, random assignment of study 
subjects to control or intervention groups, and measurements of both groups. Measurements are typically conducted 
before, and always after, the intervention. The results obtained from such studies typically yield the most definitive 
and defensible evidence of an intervention's effectiveness. 

External validity: The extent to which outcomes and findings apply (or can be generalized) to persons, objects, set- 
tings, or times other than those that were the subject of the study. 

Family: Parents (or persons serving as parents) and children who are related either through biology or through assign- 
ment of guardianship, whether formally (by law) or informally, who are actively involved together in family life — 
sharing a social network, material and emotional resources, and sources of support. 

Family in-home support: A prevention approach that addresses risk and protective factors by focusing on preserving 
families through intervention in their home environments. 

Family therapy: A prevention approach that provides professionally led counseling services to a family for the pur- 
pose of decreasing maladaptive family functioning and negative behaviors and increasing skills for healthy family 
interaction. 

Fidelity: Agreement (concordance) of a replicated program model or strategy with the specifications of the original. 
Framework: A general structure supporting the development of theory. 

Generalizability: The extent to which program findings, principles, and models apply to other populations and/or set- 
tings. 

Impact: The net effect observed within an outcome domain. 

Incidence: The number of new cases of a disease or occurrences of an event in a particular time period, usually 
expressed as a rate, with the number of cases as the numerator and the population at risk as the denominator. 
Incidence rates are often presented in standard terms, such as the number of new cases per 100,000 population. 

Indicated prevention measure: A preventive measure directed to specific individuals with known, identified risk factors. 
Individual-centered approach: A prevention approach that focuses on the problems and needs of the individual. 

Initiation: The stage at which a prevention program is underway but still on trial. Community members often have 
great enthusiasm for the effort at this stage because it has not yet encountered obstacles. 

Instrument: A device researchers use to collect data In an organized fashion, such as a standardized survey or inter- 
view protocol. 

Integrity: The level of credibility of study findings based on peer consensus ratings of quality of implementation and 
evaluation methods. 

Intended measurable outcome: The overall expected consequences and results of the interventions within each pre- 
vention approach. 
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Intervention: An activity or set of activities to which a group is exposed in order to change the group's behavior. In 
substance abuse prevention, interventions may be used to prevent or lower the rate of substance abuse or substance 
abuse-related problems. 

Methodology: A procedure for collecting data. 

Multicomponent program: A prevention program that simultaneously uses multiple interventions that target one or 
more substance abuse problems. Programs that involve coordinated multiple interventions are likely to be more 
effective in achieving the desired goals than single-component programs and programs that involve multiple but 
uncoordinated interventions. 

Nonexperimental design: A type of research design that does not include random assignment or a control group. In 
nonexperimental research designs, the attribution of an observed effect to the intervention is compromised. 

Outcome: Changes observed on targeted measures. 

Outcome evaluation: An analysis that focuses research questions on assessing the effects of interventions on intended 
outcomes. 

Parent and family skills training: A prevention approach in which parents are trained to develop new parenting skills 
and children are trained to develop prosocial skills. 

Pretests and posttests: In research designs, the collection of measurements before and after an intervention to assess 
its effects. 

Prevalence: The number of all new and old cases of a disease or occurrences of an event during a particular time 
period, usually expressed as a rate, with the number of cases or events as the numerator and the population at risk as 
the denominator. Prevalence rates are often presented in standard terms, such as the number of cases per 100,000 
population. 

Prevention principle: A principle is prescriptive and can provide implementation directions and define effective prac- 
tices. A principle can be derived from science-based program evaluations, either across multiple program implemen- 
tations of the same type or of programs of different types through meta-analyses. 

Program: The sum of all program modules implemented by an administrating agent. 

Program activity: A specified set of behaviors that constitutes a portion of an intervention strategy (e.g., lecture, field 
trip). 

Program component: The module or component is one of several parts that are grouped together to form a complete 
program. 

Program evaluation: The application of scientific research methods to assess a program's concepts, implementation, 
and effectiveness. 

Program model: A program taken as a whole. The model comprises the program activities and interventions and the 
administrative structure. 

Program module: An intervention activity affecting a target population. 

Protective factor: An influence that inhibits, reduces, or buffers the probability of drug use or abuse, or a transition to 
a higher level of involvement with drugs. 

Qualitative data: In evaluation studies, contextual information that usually describes participants and interventions. 
These data are often presented as text. The strength of qualitative data is their capability to illuminate evaluation find- 
ings derived from quantitative methods. 

Quantitative data: In evaluation studies, measures that capture changes in targeted outcomes (e.g., substance abuse) 
and intervening variables (e.g., attitudes toward substance abuse). The strength of quantitative data is their use in test- 
ing hypotheses and determining the strength and direction of effects. 

Quasi-experimental design: A research design that includes intervention and comparison groups and measurements 
of both groups, but in which assignments to the intervention or comparison groups are not done randomly. In such 
research designs, attribution of an observed effect to the Intervention is less certain than in experimental designs. 
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Random assignment: The process through which members of a pool of eligible study participants are assigned to 
either an intervention group or a control group on a random basis, such as through the use of a table of random 
numbers. 

Reliability: The extent to which a measure produces the same result time after time, across venues and raters. 

Representative sample: A segment of a larger body or population that mirrors in composition the characteristics of 
the larger body or population. 

Research: The systematic effort to discover or confirm facts by scientific methods of observation and experimentation. 

Resilience: Either the capacity to recover from traumatica I ly adverse life events (e.g., the death of a parent, divorce, 
sexual abuse, homelessness, or a catastrophic event) and other types of adversity so as to achieve eventual restora- 
tion or improvement of competent functioning or the capability to withstand chronic stress (e.g., extreme poverty, 
alcoholic parents, chronic illness, or ongoing domestic or neighborhood violence) and to sustain competent function- 
ing despite ongoing stressful and adverse life conditions. 

Risk factor: A condition that increases the likelihood of substance abuse. 

Sample: A segment of a larger body or population. 

Science-based: Substantiated through an expert consensus process. Conceptual and exact replications add to the 
credibility of findings, principles and models as being effective. (See operational definition in text for further elabora- 
tion.) 

Selective prevention measure: A preventive measure directed to subgroups of populations that have higher-than- 
average risk for developing a problem or a disorder. 

Simple random sample: In experimental research design, a sample derived by indiscriminate selection from a pool of 
eligible participants, such that each member of the population has an equal chance of being selected for the sample. 

Single-component program: A prevention strategy using a single intervention to target one or more problems. 

Sociodemographic factors: Social trends, influences, or population characteristics that affect substance abuse-related 
risks, attitudes, or behaviors. Such factors can have an indirect but powerful influence. 

Social development model: A model that seeks to explain behaviors, which are themselves risk factors for substance 
abuse, by specifying the socialization processes (the interaction of developmental mechanisms carried out through 
relationships with family, school, and peers) that predict such behavior. 

Social ecology model: A model that posits that an adolescent's interactions with social, school, and family environ- 
ments ultimately influence substance abuse and other antisocial behaviors. The model also emphasizes the impor- 
tance of increasing opportunities within the social environment for youth to develop social competencies and 
self-efficacy. 

Statistical significance: The strength of a particular relationship between variables. A relationship is said to be statisti- 
cally significant when it occurs so frequently in the data that the relationship's existence is probably not attributable 
to chance. 

Strategy: An individual component of a program intervention (e.g., life skills training or mentoring). CSAP promul- 
gates six specific strategies. These strategies are Information dissemination, prevention education, alternatives, prob- 
lem identification and referral, community-based process, and environmental strategies. 

Substance abuse: Refers to the consumption of psychoactive drugs in such a way as to significantly impair an indi- 
vidual's functioning in terms of physical, psychological, or emotional health; Interpersonal interactions; or function- 
ing in work, school, or social settings. The use of psychoactive drugs by minors Is considered substance abuse. 

Universal preventive measure: A preventive measure directed to a general population or a general subsection of the 
population not yet identified on the basis of risk factors, but for whom prevention activity could reduce the likeli- 
hood of problems developing. 

Utility: Usefulness. Any science-based finding or principle has utility if it can be used to guide program development 
or implementation. 

Validity: The extent to which a measure of a particular construct truly reflects that construct. 
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Variable: A factor or characteristic of an intervention, participant, or context that may influence or be related to the 
possibility of achieving intermediate or long-term outcomes. 

NOTE: This glossary is based in part on work performed by Birch & Davis Associates, Silver Spring, Md.; Westover 
Consultants, Silver Spring, Md.; the Pacific Institute for Research and Evaluation, Bethesda, Md.; The CDM Group, 
Chevy Chase, Md. (under contract to CSAP); and Paul Brounstein, Ph.D., and Stephen Gardner, D.S.W., CSAP. 
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